Increasing Forecast Accuracy by Mechanically Combining Human and Automated Predictions Using a Knowledge Based System

There is an increasing interest in the question of what might be the appropriate future role for the human in the forecast process. It is asserted that computergenerated forecasts are unable (by themselves) to fully replicate the decision-making processes of human forecasters. Similarly, it is also asserted that human forecasters are unable (by themselves) to optimally integrate into the forecasting process, guidance from computer-generated predictions. However, there is the accepted mathematical concept that two or more inaccurate but independent predictions of the same future events may be combined to yield predictions that are, on the average, more accurate than either of them taken individually. Automated and human forecasts might be expected to "bring to the table" different knowledge sets, and this suggests the development of a weather forecasting system that mechanically combines human and computer-generated predictions. __________________________________ *Corresponding author address: Harvey Stern, Bureau of Meteorology, Box 1636, Melbourne, Vic., 3001, Australia; e-mail: [email protected] About the author: Dr Harvey Stern holds a Ph. D. from the University of Melbourne, for a dissertation entitled Statistically based weather forecast guidance. Dr Stern also holds a Graduate Diploma in Applied Finance and Investment from the Securities Institute of Australia. His 1992 paper, The likelihood of climate change: a methodology to assess the risk and the appropriate defence, which explored the application of options pricing theory to climate change, was the first paper to be published on what later became known as weather derivatives. This paper reports on the evaluation of a knowledge based system, modified in order to mechanically combine human and computer-generated predictions. The system’s output is firstly evaluated over a “realtime” trial of 100 days duration. The trial reveals that forecasts generated by mechanically combining the predictions explain 7.7% additional variance of weather (rainfall amount, sensible weather, minimum temperature, and maximum temperature) over that explained by the human (official) forecasts. In the light of the results of the 100-day trial, a number of minor modifications are made to the system and the trial is then continued. After 365 Day-1 to Day-7 forecasts, that is, 2555 individual predictions, the average lift in percentage variance of weather explained is 7.9% over that explained by the current official forecasts. With computer-generated forecasts unable to fully incorporate human forecasters’ valuable domain and contextual knowledge, there should be a need for the human forecaster well into the future. That future role may be as an input to a system that mechanically combines human predictions with computergenerated forecasts.